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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 

WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )ED Responsive to communication(s) filed on 30 June 2006 . 
2a)D This action is FINAL. 2b)[3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-20,26 and 27 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) I3 Claim(s) 1-20, 26-27 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Continued Examination Under 37 CFR LI 14 

1 . A request for continued examination under 37 CFR 1.114 was filed in this application 
after appeal to the Board of Patent Appeals and Interferences and the Examiner's Answer (dated 
03/21/06), but prior to a decision on the appeal. Since this application is eligible for continued 
examination under 37 CFR 1.1 14 and the fee set forth in 37 CFR 1.17(e) has been timely paid, 
the appeal has been withdrawn pursuant to 37 CFR 1.114 and prosecution in this application has 
been reopened pursuant to 37 CFR 1.114. Applicant's submission filed on 06/30/06 has been 
entered. 

Remark 

2. Claims 3, and 21-25 have been cancelled, and claims 26-27 have been added; pending 
claims 1-2, 4-20, and 26-27 are for reconsideration. 

Claim Rejections - 35 USC 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 
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4. Claims 1-2, 4-10, 17-20, and 26-27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hjelsvold et al. (U.S. Patent No. 6,546,555 Bl) in view of Nagasaka et al. 
(U.S. Patent 6,400,890 Bl). 

Regarding claim 1, Hjelsvold discloses "a method for processing video, the method 
comprising: displaying a sequence of video segments at a display of a user, determining an 
association between a first video segment including a particular feature and at least one 
additional information source also including that feature; and utilizing the association to display 
information from the additional information source based at least in part on a selection by a user 
of the feature in the first video segment while one or more video segment are displayed to the 
user", i.e., one or more video segments are delivering and displaying to the viewer, while 
viewing the programming segment with a particular feature, the viewer further access to related 
information to that feature from an additional information source of a vendor for that particular 
product or service based on the defined association between the video segment and related 
information sequences (Figs. 16-18, and col. 2/line 58 to col. 3/line 32 for hypervideo to link to 
additional information related to a feature of a product or a service; and col. 1 1/line 16 to col. 
1 2/line 10 for further details on the determination of association between parameter values and 
hyperlink and hypervideo). 

Hjelsvold does not clearly suggest the step of "extracting a feature from one or more 
video segments of the sequence" and then "defining a link between the feature and the at least 
one additional information source to facilitate a display of information" (as pre-amended); 
however, Nagasaka teaches a same technique of extracting a feature from one or more video 
frame sequences, and similar features can be stored, indexed, and searched as additional 
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information related to the feature(s) can be defined (Nagasaka, Figs. 2, 9, 10A-10B, 15 & 16, and 
col. 2/lines 27-60 for details on additional information related to the extracted features from one 
or more video sequences are defined, associated them to each other for searching, retrieving, and 
displaying to the user; and col. 4/line 57 to col. 5/line 32 for a frame feature extractor; and Fig. 
19 and col. 1 5/line 40 to col. 16/line 34 for similar features are gathered, defined and associated 
from features being extracted from one or more video sequences). Therefore, it would have been 
obvious to one of ordinary skill in the art to modify Hjelsvold's system with Nagasaka's teaching 
technique of feature extraction as noted in order to provide the user a sequence of video 
segments, extracting a feature from one or more video segments, defining a link between the 
feature and the at least one additional information related to the feature as preferred. 

As for claim 2, in view of claim 1, Hjelsvold and Nagasaka discloses "wherein defining 
the link includes retrieving the link from a memory based on an identification of the feature" 
(Hjelsvold, Fig. 1, for the server 10 retrieves the meta-data from a database for filtering based on 
related features as explained earlier; and Nagasaka, Fig. 3 for features with identification). 

(Claim 3 has been cancelled). 

As for claim 4, in view of claim 1, Hjelsvold further discloses "wherein the additional 
information source includes an additional video segment that also includes the feature" (as 
already discussed in claim 1). 

As for claims 5 and 6, in view of claim 4, Hjelsvold discloses "switching from display of 
the first video segment to display of the additional video segment" (as shown in Figs. 16-18, for 
the display of the next screen including the video feature of the feature of a product or service); 
and Hjelsvold further discloses "displaying the additional video segment at least in part in a 
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separate portion of a display which also includes at least a portion of the one or more video 
segment" (Figs. 17-18, and col. 12/lines 1-33). 

As for claim 7, in view of claim 1, Hjelsvold further discloses "wherein the feature 
includes a video feature", i.e., the selected target is at least one frame of the video segment as a 
window frame of image (Fig. 17 and col. 12/line 1-33 clearly show the next additional 
information is extracted from at least one frame of the video sequences, as discussed earlier in 
Figs. 4-5 for the building of an association between parameter values for video sequences within 
the filtering process). 

As for claim 8, in view of claim 7, Hjelsvold discloses "wherein the video feature 
includes at least one of a frame characterization, a face identification, a scene identification, an 
event identification, and an object identification" (Figs. 14, and 16-18 for these features). 

As for claims 9 and 10, in view of claim 1, Hjelsvold further discloses "wherein the 
feature includes an audio feature" and "including combining an audio signal corresponding to the 
audio feature with an audio signal associated with the first video segment", i.e., as the user 
selects a desired target, the video segment including audio tracks related to the selected portion 
of feature is provided (col. 4/lines 51-64 for multimedia includes video, audio tracks and other 
objects). 

As for claim 17, in view of claim 1, Hjelsvold and Nagasaka do not discloses "wherein 
determining the association includes determining a similarity measure and a clustering 
technique"; however, this limitation is admitted as prior art by the Applicant (page 9, line 19 to 
page 10/line 13). Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Hjelsvold' technique with a known prior art using 
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similarity measure and a clustering technique for determining the association or the relationship 
in the determining step of claim 1, for the purpose of providing same information to a group of 
users with similarity interests on a certain product or service as preferred. 

As for claims 18-20, these claims with same limitations are rejected for the reasons given 
in the scope of claim 1 above in view of Hjelsvold and Nagasaka as already discussed in details 
above, to avoid unnecessary repetitions herein. 

As for claims 26-27, Hjelsvold teaches these features of storing the link and combining 
the link and the video segment to create a hyperlinked video segment (Fig. 1, and refer again to 
claims 1-2 and claims 7-9). 

5. Claims 11-16 are rejected under 35 U.S.C. 103(a) as being unpatentable over Hjelsvold et 
al. (U.S. Patent No. 6,546,555 Bl) in view of Nagasaka (US Patent 6,400,890 Bl) and Jain et al. 
(U.S. Patent No. 6,463,444 Bl). 

Regarding claim 1 1, in view of claim 9, Hjelsvold and Nagasaka do not further disclose 
"converting an audio signal corresponding to the audio feature into a textual format that is 
displayed with the first video segment"; however, such a technique of converting audio signal to 
a textual format or speech-to-text feature is known in the art. In fact, Jain, in a video cataloger 
system for providing video/audio information data to the user, teaches to use a closed caption 
decoder (Fig. 3) or speech-to-text converting technique for providing a textual format to display 
with the video to the user (Fig. 9, item 518, and col. 9/line 45 to col. 10/line 38 for audio feature 
extractors, and col. 20/lines 45-48 for speech-to-text). Therefore, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify Hjelsvold and 
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Nagasaka's system with Jain's teaching technique as disclosed in order to provide an additional 
feature such as a textual format in additional to the display of video presentation. This technique 
is helpful for some people have difficulty in hearing, so that they can read the texts on the 
display screen instead, which serves also as a motivation for modifying Hjelsvold regarding this 
limitation. 

As for claim 12, in view of claims 9 and 1 1 above, Jain further including "separating at 
least a portion of the video segment into audio categories including one or more of single- voice 
speech, multiple voice speech, music, silence and noise in order to extract the audio feature 
therefrom" (see Fig. 6 and col. 9/line 45 to col. 10/line 38 for a monitoring screen for separating 
a portion of video segment into audio categories and audio feature extractors as addressed). 

As for claim 13, in view of claims 9 and 1 1 above, Jain teaches "wherein the audio 
feature includes at least one of a music signature extraction, a speaker identification, and a 
transcript extraction", i.e., music, and/or speaker ID, signatures or sample speeches of individual 
speaker or transcripts from the speaker are within audio feature addressed (see col 9/line 18-col. 
10/line 38). 

As for claim 14, in view of claim 1, the combination of Hjelsvold and Jain teaches 
"wherein the feature is a textual feature", i.e., applied Jain's technique of textual feature 
extracted, the at least one frame of the video segment as discussed earlier of Hjelsvold would 
contain the textual feature (see claims 1, 7 and 11). 

As for claim 15, in view of claim 14, Jain further discloses "displaying information 
corresponding to the textual information as an overlay on a display of the first video segment" 
(as illustrated in Fig. 17, and col. 14/lines 15-63). 
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As for claim 16, in view of claims 1 and 14, Jain further teaches "the feature includes at 
least one multi-dimensional feature vector extracted from a portion of the video segment using a 
feature extraction technique" (Fig. 14, and col. 12/lines 20-46 for feature extraction technique 
addressed). 



Conclusion 

6. Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 
or faxed to PTO New Central Fax number: 

(571) 273-8300, (for Technology Center 2600 only) 

"Hand dedveries must 6e made to Customer Service Window, 
<Rando[ph (Building, 401 (Dutany Street, jlkj&ndria, VjL 22314. 

7. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Krista Kieu-Oanh Bui whose telephone number is (571) 272-7291. The 
examiner can normally be reached on Monday-Friday from 9:30 AM to 7:00 PM, with alternate 
Fridays off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W. Miller, can be reached at (571) 272-7353. 

The Art Unit location of your application in the USPTO has changed. To aid in 
correlating any papers for this application, all further correspondence regarding this application 
should be directed to Division or Art Unit 2623. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




Kieu-Oanh Bui 
Primary Examiner 
Art Unit 2623 
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August 17, 2006 



